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Abstract 

An  approximating  algorithm  for  computing  equivalent  degrees  of  freedom  of  the  modified  Allan 
variance  and  its  square  root,  the  modified  Allan  deviation  (MVAR  and  MDEV),  and  the  time  variance 
and  time  deviation  (TKiR  and  TDEV)  is  presented,  along  with  an  algorithm  for  apprmtimating 
the  inverse  chi-squared  distribution.  These  two  algorUhms  allow  relatively  simple  computations  of 
con^dence  intervals  on  MDEV  and  TDEV  the  latter  currency  used  as  a  standard  in  ^  telecom¬ 
munications  industry.  These  algorithms  enable  users  to  present  variance  results  with  confidence 
intervals  corresponding  to  any  useful  probability  for  most  data  lengths  and  noise  types. 


1  INTRODUCTION 

We  present  here  a  simplified  algorithm  for  calculating  approximate  confidence  intervals  on  the 
modified  Allan  deviation,  MDEV,  and  therefore  also  on  the  related  time  deviation  TDEV,  The 
algorithm  has  two  parts:  the  first  gives  approximate  equivalent  degrees  of  fireedom,  edf,  for 
the  fully  overlapped  estimate  of  MVAR;  the  second  gives  approximate  values  of  the  inverse 
chi-squared  distribution.  An  algorithm  for  estimating  edf  for  the  other  measure  commonly  used 
in  time  and  frequency  metrology,  the  original  Allan  deviation,  was  published  previously.!!! 

Confidence  intervals  are  defined  in  terms  of  edf  and  the  chi-squared  distribution  as  follows. 
If  s*  denotes  the  usual  sample  variance  of  n  independent  and  identically  distributed  Gaussian 
measurements  (white  noise)  with  actual  variance  then 

‘CoatributionB  of  the  U.S.  govermnent  are  not  subject  to  copyright. 
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(1) 


has  a  chi-squared  distribution  with  v  —  n-i  degrees  of  freedom. In  the  classical  situation, 
the  number  of  degrees  of  freedom  associated  with  <7*  is  an  integer  value  depending  only  on  the 
number  of  measurements,  and  exact  confidence  limits  on  the  measurement  variance  are  easily 
calculated  using  percentiles  of  the  appropriate  chi-squared  distribution.  For  example.  Figure  1 
shows  the  chi-squared  distribution  with  10  degrees  of  freedom,  and  also  depicts  the  percentiles 
a  and  b  that  are  needed  to  calculate  uncertainty  bounds  on  at  the  0.95  confidence  level  from 
a  particular  based  on  11  Gaussian  measurements. 

A  95%  confidence  interval  is  obtained  as  follows.  First,  we  find  values  a  and  b  such  that  the 
probability  is  0.95  that  U  of  Eq.  (1)  lies  between  a  and  b.  This  condition  is  equivalent  to  each 
of  the  following  inequalities: 

a  <  —  ■  y  <  6; 

0-2 


(2) 


b  a 

The  lower  and  upper  bounds  in  the  final  inequality  are  confidence  limits  on  the  unknown 
variance  cr^.  Note  that  the  confidence  factors  v/b  and  v/a  needed  in  the  calculations  are 
independent  of  the  actual  data.  They  give  the  magnitude  of  the  confidence  interval  as  a 
function  of  the  number  of  points  used  to  compute  the  variance.  Hence,  we  can  compute  these 
confidence  factors  for  various  data  lengths.  The  factors  1  —  v/b  and  v/a  —  l  give  the  multipliers 
for  the  magnitude  of  the  lower  and  upper  confidence  intervals  on  the  variances,  respectivety. 
For  deviations  such  as  TDEV,  the  corresponding  multipliers  are  1  -  and  y/v/a  - 1. 

Since  the  common  time  and  frequency  stability  measures  (AVAR,  MVAR,  TVAR)  are  calculated 
from  data  arising  from  non-white  noise  processes,  the  confidence  limit  procedure  outlined  above 
is  an  approximate  methodic]  that  is  based  on  approximating  the  distribution  of  in  Eq.  (1) 
with  the  chi-squared  distribution  with  degrees  of  freedom 


jMl 

Vor(a*)’ 


(3) 


where  a’  represents  the  appropriate  stability  measure,  TVAR,  for  example,  a®  represents  its 
corresponding  estimator,  and  Var  (a*)  is  the  variance  of  the  a’  estimator.  The  quantity  v,  which 
now  depends  on  the  noise  type,  is  called  the  equivalent  degrees  of  freedom,  edf,  since  it  need 
not  be  integer-valued. 

In  this  contribution  we  have  combined  a  previously  published  edf  approximation  algorithm  1^1 
with  an  algorithm  for  approximating  the  inverse  of  the  chi-squared  distribution  function.  The 
latter  algorithm  is  based  on  work  of  Barnes  used  in  deriving  tables  in  [5|,  but  not  published, 
and  formulas  from  Abramowitz  and  Stegun  (A&S).W  Previously,  tables  for  confidence  of 
TDEV  and  MDEV  were  published  in  [7].  These  are  exact  computations  for  edf  and  the 
associated  confidence  intervals  for  various  cases  in  computing  TDEV  and  MDEV.  We  compare 
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values  approximating  the  exact  edf  and  confidence  factors  in  tables  in  [7],  finding  a  worst 
case  disagreement  of  -9.7%  for  the  edf  and  +10.8%  for  the  confidence  intervals.  Most  cases 
are  much  better  than  that.  The  confidence  intervals  are  pessimistic  if  they  arc  too  large  and 
optimistic  if  they  are  too  small.  In  many  cases  here,  pessimism  is  better  than  optimism,  since 
the  true  value  of  the  variance  is  more  certain  to  lie  in  a  larger  range  than  a  smaller.  For  the 
comparison  with  the  published  tables  the  confidence  intervals  are  no  smaller  than  -3.3%. 


2  APPROXIMATION  FOR  EQUIVALENT  DEGRESS  OF 
FREEDOM 

This  version  of  the  formula  is  restricted  to  the  case  of  the  usual  fully  overlapped  estimator  of 
MVAR  or  TVAR  ([8],  Eq.  (12);  [4],  Eq.  (6),  m,  =  1). 

Let 

N  ~  number  of  time  residuals, 
m  =  averaging  time  /  sample  period, 

M  =  N -Sm  +  l,  the  number  of  terms  summed  in  the  estimate, 
q  —  M/m. 


Restrictions; 

i\r>  16, 
m  <  N/5. 


The  approximate  edf  is  given  by 


e4f  = 


aoq 

l-oi/g’ 


where  oo  and  O]  are  given  in  Ikble  I  as  fimctions  of  m  and  the  noise  type. 


(4) 


Table  I.  CoefiBcients  for  Approximate  edf  Calculation 


Noise  '^pe 

m  = 

Oo 

1 

ai 

m  = 

Oo 

2 

Oj 

m  >  2 

Oo  Oj 

White  PM 

0.514 

"IT 

0.935 

0 

1.225 

0.589 

Flicker  PM 

0.576 

0 

0.973 

0 

1.003 

0.602 

White  FM 

0.667 

0 

1.010 

0 

0.968 

0.571 

Flicker  FM 

0.811 

0 

1.027 

0 

0.947 

0.416 

Random-Walk  FM 

1.000 

0 

0.866 

0 

0.768 

0.411 

Under  the  assumptions  given  above,  a 
observed.  Usually,  it  is  much  less. 

maximum  error  of 

11.1%  in  this 

approximation  has  been 
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3  APPROXIMATION  FOR  INVERSE  OF  CHI-SQUARED 
DISTRIBUTION 


Let  17  be  a  chi-squared  random  variable  with  v  degrees  of  freedom  {v  can  be  nonintegral). 
Let  0  <  p  <  1.  Define  x  =  i(p,  v)  as  the  lOOp  percentile  of  the  distribution  of  V\  thus  p  is  the 
probability  that  17  <  i.  The  ^gorithm  given  below  computes  an  approximation  to  x. 

Restrictions: 

0.005  <p<  0.995. 

Maximum  observed  error  with  these  restrictions:  3% 


if  p  <  5  and  i/  <  10  then 

!  Method:  truncate  power  series  in  A&S  [6]  26.4.6,  invert  by  iteration 

a  =  I//2 

!  Calculation  of  G  =  r(H-o)  (A&S  6.1.35) 

constants;  a  =  -0.5748646,  =  0.9512363,  cs  =  -0.6998588,  c*  =  0.4245549,  cg  = 

-0.1010678 

n  =  integer  part  of  a 
y  =  o-n 

G  =  1  -I-  cip  +  csy®  -I-  csy’  +  ay*  +  Csy® 
for  A:  =  1  to  n  !  Do  nothing  if  n  =  0 
G  =  G(y-|-*) 
next  k 
A  =  pG 
u  =  0 

for  t  =  1  to  7 


next  i 
X  =  2u 


!  Method:  A&S  26.4.17 
Pi  =inin(p,l-p) 

!  Calculation  of  =  inverse  of  normal  distribution  at  1  -pi  (A&S  26.2.22) 
constants;  oq  =  2.30753,  oi  =  0.27601,  bi  -  0.99229,  *2  =  0.04481 
t  =  v7— 21np, 
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X  =  t  —  (oq  +  ffllt)  /  (l  +  +  ^2^*) 

a  =  aignum  (p  —  5)  !  signum  (a)  =  1  if  u  >  0,  —1  if  u  <  0,  0  if  «  =  0 

b  =  2/  (9i/) 

a:  =  V  ^1  —  b  +  aXV^ 


4  A  NUMERICAL  EXAMPLE 

Before  giving  tabular  results,  we  show  by  example  how  they  are  used  and  how  they  are  calculated 
by  the  algorithms  given  above.  Assume  the  situation  of  the  last  line  of  Thble  II  below:  white 
PM  noise,  1,Q2S  time  residuals,  and  averaging  time  =  128  sample  periods.  Suppose  that  an 
MDEV  value  a  is  computed  by  a  fully  overlapped  estimate.  The  tabulated  95%  lower  and  upper 
factors  are  33.89%  and  104.1%.  Therefore,  a  95%  confidence  interval  for  the  true  MDEV  is 
0.661  a  to  2.041  a. 

The  tabulated  edf  and  confidence  factors  are  obtained  as  follows:  N  =1,025,  m  =  128,  M  = 
1025-3x128+1  =  642  (the  number  of  summands  in  the  estimate),  q  =  M/m  =5.0156,  ao  = 
1.225,  ai  =  0.589  from  Thble  I,  edf  =  6.9617  from  Eq.  (4).  For  95%  confidence  we  need  to 
compute  the  2.5%  and  97.5%  chi-squared  levels.  The  inverse  chi-squared  algorithm,  with  = 
6.9617  and  p  =  0.025,  gives  x  =  1.6720  as  the  2J%  level,  denoted  by  a  in  (2).  Similarly, 
the  97.5%  level  is  15.928,  denoted  by  b.  The  computed  confidence  factors  are  1  -  y/vjb  = 
0.3389,  \/t^-  1  =  1.0405.  (Note  that  the  values  in  Table  II  were  computed  from  values  of  oo 
and  Si  having  more  significant  digits  than  the  ones  given  in  Table  I.) 


5  RESULTS 

The  data  in  the  tables  are  the  results  for  white  PM  with  fully  overlapped  estimates.  Table  II 
gives  the  approximate  edf  and  confidence  factors.  Ikble  III  gives  the  percentage  errors  from 
the  exact  values  as  found  in  [7].  The  errors  for  white  PM  are  the  largest  of  the  various  noise 
types. 
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Table  n.  Approximate  edf  and  Confidence  Factors 


Noise  Type:  White  PM 


N 

m 

edf 

lower  68% 

upper  68% 

lower  95% 

upper  95% 

17 

1 

7.714 

17.74 

39.14 

32.79 

94.61 

17 

2 

5.610 

19.67 

50.41 

36.24 

128.7 

33 

1 

15.94 

13.65 

23.38 

25.52 

52.42 

33 

2 

13.09 

14.71 

26.67 

27,40 

60.97 

33 

4 

7.543 

17.87 

39.82 

33.03 

96.58 

65 

1 

32.40 

10.29 

14.96 

19.46 

31.99 

65 

2 

28.05 

10.91 

16.33 

20.60 

35.19 

65 

4 

17.29 

13.23 

22.17 

24.78 

49.37 

65 

8 

7.241 

18.12 

41.10 

33.47 

100.3 

129 

1 

65.31 

7.622 

9.916 

14.58 

20.63 

129 

2 

57.97 

8.030 

10.62 

15.33 

22.17 

129 

4 

36.86 

9.746 

13.84 

18,48 

29.42 

129 

8 

16.98 

13.33 

22.43 

24.94 

50.03 

129 

16 

7.091 

18.24 

41.78 

33.69 

102.3 

257 

1 

131.1 

5.579 

6.715 

10.76 

13.74 

257 

2 

117.8 

5.857 

7.123 

11.28 

14.60 

257 

4 

76.04 

7.128 

9.095 

13.66 

18.84 

257 

8 

36.55 

9.780 

13.91 

18.54 

29.58 

257 

16 

16.83 

13.37 

22.56 

25.02 

50,36 

257 

32 

7.016 

18.31 

42.13 

33.80 

103.3 

513 

1 

262.8 

4.045 

4.610 

7.856 

9.332 

513 

2 

237.5 

4.241 

4,867 

8.229 

9.864 

513 

4 

154.4 

5.177 

6.141 

10.00 

12.53 

513 

8 

75.73 

7.141 

9.116 

13.68 

18.89 

513 

16 

36.40 

9.798 

13.95 

18.57 

29.66 

513 

32 

16.75 

13.40 

22.63 

25.07 

50.53 

513 

64 

6.978 

18.34 

42.30 

33.86 

103.8 

1025 

1 

526.1 

2.913 

3.195 

5.685 

6.421 

1025 

2 

476.9 

3.052 

3.363 

5.954 

6.766 

1025 

4 

311.1 

3.737 

4.214 

7.267 

8.513 

1025 

8 

154.1 

5.182 

6.148 

10.01 

12.54 

1025 

16 

75.58 

7.148 

9,126 

13.69 

18.91 

1025 

32 

36.32 

9.806 

13.97 

18.59 

29.70 

1025 

64 

16.71 

13.41 

22.67 

25.09 

50.62 

1025 

128 

6.959 

18.36 

42.39 

33.89 

104.1 
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rbble  ni.  Percentage  Errors  100  (^pproonmate  —  Correct)  }Carre<A 


Noise  *^0:  White  PM 


w 

m 

edf 

lower  68% 

upper  68% 

lower  95% 

upper  95% 

17 

1 

-3.4 

0.2 

2.2 

1.0 

3.2 

17 

2 

-9.7 

2.2 

8.4 

3.0 

10.8 

33 

1 

-1.6 

-0.1 

0.2 

0.5 

1.4 

33 

2 

-4.2 

0.8 

2.0 

1.4 

3.7 

33 

4 

3.4 

-2.1 

-2.9 

-1.2 

-3.1 

65 

1 

-0.8 

-0,3 

-0.2 

0.2 

0.5 

65 

2 

-2.0 

0.1 

0.5 

0.6 

1.3 

65 

4 

3.9 

-2.2 

-3.3 

-1.5 

-2.6 

65 

8 

-3,6 

0.3 

2.4 

1.0 

3.4 

129 

1 

-0.4 

-0.4 

-0.4 

0.0 

0.1 

129 

2 

-1.0 

-0.2 

-0.1 

0.3 

0.5 

129 

4 

4.0 

-2.3 

-3.0 

-1.7 

-2.6 

129 

8 

-2.9 

0.4 

1.1 

1.0 

2.3 

129 

16 

-5.3 

0.9 

3.9 

1.6 

5.2 

257 

1 

-0.2 

-0.5 

-0.5 

-0.1 

-0.1 

257 

2 

-0.5 

-0.4 

-0.3 

0.0 

0.1 

257 

4 

4.1 

-2.4 

-2.9 

-1.9 

-2.5 

257 

8 

-2.7 

0.5 

1.0 

1.0 

1.7 

257 

16 

-4.6 

1.1 

2.3 

1.6 

3.6 

257 

32 

-5.8 

1.0 

4.3 

1.8 

5.7 

513 

1 

-0.1 

-0.5 

-0.5 

-0.1 

-0.1 

513 

2 

-0.2 

-0.5 

-0.5 

0.0 

0.0 

513 

4 

4.1 

-2.4 

-2,8 

-1.9 

-2.4 

513 

8 

-2.6 

0.6 

0.9 

1.0 

1.4 

513 

16 

-4.4 
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95%  Confidence 


Figure  1  Finding  the  95%  confidence  limits  under  the  chi- 
squared  distribution  with  10  degrees  of  fi-eedom. 
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Questions  and  Answers 

JOHN  DICK  (JPL):  Where  does  the  number  68  percent  come  from?  And  what  do  you 
recommend  as  confidence  limits  in  our  use  of  error  bars  on  Allan  deviation  plots? 

MARC  WEISS:  Sixty-eight  percent  is  a  typical  one  sigma  land  of  number.  When  you  take  a 
standard  deviation,  it’s  typically  a  one  sigma  kind  of  number.  Sixty-eight  percent  probability  is 
one  sigma  on  a  normal  distribution;  that’s  where  that  came  from.  Ninety-five  percent  is  two 
sigma.  Three  sigma  is  the  99  percent,  I  believe. 

So  that’s  where  I  chose  those.  Now  which  one  to  use  depends  on  the  application  and  how 
much  you’re  willing  to  be  wrong.  If  two  times  out  of  three  is  good  enough,  then  one  sigma’s 
good  enough.  But  if  you  want  it  to  never  fail,  three  sigma  is  not  good  enough. 

I  think  that’s  a  very  important  question.  In  our  ^sterns,  we  really  need  to  think  about  what 
happens  if  they  fail;  and  how  much  do  we  want  to  avoid  that;  and  how  much  are  we  willing  to 
pay  for  it.  And  when  you  write  a  spec,  another  big  question  is,  “Should  the  confidence  bars  be 
included  in  the  spec?”  Should  you  write  a  spec  that  says,  “The  Allan  variance  shall  not  exceed 
this  number  with  a  95  percent  probability?”  There  are  problems  with  that  because  if  you  do 
that,  that’s  going  to  change  how  long  you  test  the  system  in  order  to  show  it  meets  the  spec. 

ROBERT  DOUGLAS  (NRC,  CANADA):  In  calculating  the  exact  values,  we  presumably  are 
using  a  Gaussian  normal  distribution  so  you  can  calculate  the  second  moments.  Is  that  right? 

MARC  WEISS:  Do  you  want  to  answer  that.  Chuck? 

CHARLES  GREENHALL  (JPL):  Yes,  we’re  assuming  the  second  differences  of  phase  are 
normal. 

ROBERT  DOUGLAS:  Then  my  question  is  do  you  have  any  recommended  procedures  for 
checking  that  this  is  in  fact  true  in  our  datasets? 

CHARLES  GREENHALL:  No.  There  are  tests  for  normal  distribution. 


